Perspectives from Journal Leaders
Evan D. Kharasch, MD PhD
Editor-in-Chief
Merel H. Harmel Professor of Anesthesiology
Vice-Chair for Innovation
Department of Anesthesiology
Director of Research Entrepreneurship
Duke University School of Medicine
Durham, NC 27710
Trusted Evidence
Readers: Peer-reviewed content the most important source
of information to support clinical practice or inform research
Most common reason to read ANESTHESIOLOGY? Authoritative
source of clinical practice & research information
Highest rated ANESTHESIOLOGY attribute: Credibility of content
Discovery to practice
Interests and content of ANESTHESIOLOGY
Span from discovery to practice
Translate discovery into practice
Influence science from discovery to practice
Trusted Evidence: Discovery to Practice
Trusted Evidence:
Transparency of reporting
Peer review imperative
Soundness of results
Validity of conclusions
Scholarly integrity
Richness of content
Reach and readability
Original investigations
Perioperative Medicine
Critical Care Medicine
Pain Medicine
Concepts:
1. Because of many threats to interpretations of observational research, a
key element is development of a carefully crafted plan of analysis that
prospectively defines variables (primary outcome, exposure variables,
confounders), subgroup analyses, sensitivity analyses, selection and
adjustment methods, to reduce impact of bias on interpretation of results
2. Development of the analysis plan is best done before examination of the
data, when the study is being designed
3. Failure to pre-specify an analysis plan can lead to unwanted consequences
Reporting of Observational Research in ANESTHESIOLOGY:
The Importance of the Analysis Plan
James C. Eisenach, M.D., Sachin Kheterpal, M.D., Timothy T. Houle, Ph.D.
Anesthesiology 2016;124:998–1000
Changes in Journal policy regarding observational studies:
1. Strongly encourage authors of observational studies to develop a
statistical analysis plan before accessing data
2. Strongly encourage reviewers and readers to consider these plans
when evaluating the reliability of their conclusions
3. Require going forward that authors transparently report in the article
whether an analysis plan was developed before accessing data
Reporting of Observational Research in ANESTHESIOLOGY:
The Importance of the Analysis Plan
James C. Eisenach, M.D., Sachin Kheterpal, M.D., Timothy T. Houle, Ph.D.
Anesthesiology 2016;124:998–1000
Concepts:
o Author participation in the 2016
requirements and their enforcement
has been inconsistent
o Observational research continues to
evolve and grow; so too should
expectations of readers in
transparency of methods & reporting
Observations and Observational Research
Evan D. Kharasch, M.D., Ph.D.
Anesthesiology 2019;131:1–4
Policy:
Observational studies should report according to
the STROBE group guidelines (Strengthening
Reporting of Observational Studies in
Epidemiology)
ANESTHESIOLOGY will require explicit statement
in manuscripts of whether a data analysis and
statistical plan was defined before accessing data
Observations and Observational Research
Evan D. Kharasch, M.D., Ph.D.
Anesthesiology 2019;131:1–4
Methods section must state: A data analysis and statistical plan was
(1) written and posted on a publically accessible server
(ClinicalTrials.gov or other) before data were accessed,
(2) written and filed with private entity (IRB or other) before data were
accessed,
(3) written, date-stamped (permanent dated electronic signature), and
recorded in the investigators’ files before data were accessed, or
(4) written after the data were accessed
If there was an a priori data analysis and statistical plan (#1-3), authors are requested and strongly
encouraged to include the plan as supplemental digital content at the time of initial manuscript submission
Observations and Observational Research
Evan D. Kharasch, M.D., Ph.D.
Anesthesiology 2019;131:1–4
• ANESTHESIOLOGY will not presently require
registration of observational studies
• Observational studies occasionally may need to add
post hoc analyses. Such analyses outside the a priori
analysis plan must be interpreted appropriately, and
can occasionally lead to important discoveries
• However, authors must transparently explain the
approach and justify why it was done
Observations and Observational Research
Evan D. Kharasch, M.D., Ph.D.
Anesthesiology 2019;131:1–4
July 18, 2019
Observational studies:
• Must submit the prespecified SAP with the manuscript
• Must deposit SAPs in an online repository
• Must use accepted prespecified methods to control false discovery
• If no prespecified analysis plan exists, Methods section should
provide the planned method of analysis
Trusted Evidence:
Transparency of reporting
Peer review imperative
Richness of content
Soundness of results
Validity of conclusions
Reach and readability
Title
Abstract
Hypothesis
Purpose
Methods
1º/2º outcome
Results
Narrative
Conclusions
Statistical analysis
False precision
Nonstandard abbreviations
Spin
Peer review imperative
Speed & quality of peer review
Author participation
Minimize revisions
1
MPOG 2019
10.18.2019
bja@med.cornell.edu
@HughHemmings
Perspectives on Multicenter Pragmatic
and Observational Research:
British Journal of Anaesthesia
Hugh C Hemmings Jr MD PhD FRCA
Editor-in-Chief, British Journal of Anaesthesia
Senior Associate Dean for Research
Professor and Chair of Anesthesiology
Weill Cornell Medicine, Cornell University
NewYork-Presbyterian Hospital
New York NY USA
3
MPOG 2019
International board
Country Accepted
United Kingdom 84
United States 81
Netherlands 29
Australia 26
France 20
Germany 17
Canada 16
Switzerland 10
Denmark 9
Sweden 9
Korea, Republic of 9
4
MPOG 2019
Observational research in the BJA
1. Prospective audits
2. Recent observational papers from the BJA:
Retrospective analysis of large data sets
5
MPOG 2019
UK National Audit Project
• Royal College of Anaesthetists NAP program since 2003
• Last four have had significant clinical impact in anesthesia:
– NAP3: major complications of central neuraxial blocks (2009)
– NAP4: serious adverse events related to airway management
in the OR, ICU and ED (2011)
– NAP5: awareness under general anesthesia (2014)
– NAP6: perioperative anaphylaxis (2018)
– NAP7: perioperative cardiac arrest (ongoing)
• Consist of: baseline survey as benchmark,
activity survey of current practice, high fidelity
national registry of rare events over a year,
analysis and recommendations
6
MPOG 2019
Audits as observational
research and
implementation science
• Research across the translational
spectrum.
• Rigorous audits fall under the umbrella of
translational research (T2-T4)
o T2 - High quality surveys and
observational studies
o T3 – Dissemination and
implementation of findings from audit
process
o T4 – Guidelines and standards
7
MPOG 2019
NAP6 series
• Pros: national coverage of UK with
involvement of most clinicians
• Cons: under-reporting of
complications, impact on changes in
practice (post hoc ergo propter hoc
fallacy); NHS conducive and not
available in other countries
Special Issue on Perioperative
Anaphylaxis July 2019
8
MPOG 2019
NAP6 series
• Pros: national coverage of UK with
involvement of most clinicians
• Cons: under-reporting of
complications, impact on changes in
practice (post hoc ergo propter hoc
fallacy); NHS conducive and not
available in other countries
Special Issue on Perioperative
Anaphylaxis July 2019
9
MPOG 2019
10
MPOG 2019
11
MPOG 2019
Retrospective analysis of large data
sets
• Observational epidemiological studies
• Move toward preplanned analyses
• Mostly hypothesis generating
BJA Instructions:
Observational studies: STROBE statement reporting
checklists must be provided. Please check the STROBE statement
website for up-to-date information concerning specific study
types, including observational studies in epidemiology, genetic
association studies, cohort studies, case-control studies, and
cross-sectional studies.
12
MPOG 2019
13
MPOG 2019
14
MPOG 2019
15
MPOG 2019
16
MPOG 2019
17
MPOG 2019
Three examples from the
November 2019 issue
17
18
MPOG 2019
Three papers in Quality section
19
MPOG 2019
• Observational cohort study
• Large Danish registry
• Showed that DAH30 was as
good as DAH90 in detecting
compications
• Lower in high-risk patients
• More medical than surgical
complications
20
MPOG 2019
• Retrospective single center
analysis
• Association of any
complication with caseload
• No association with
anesthesiologist case
volume or experience
• Reduced complication with
surgeon case volume >50
per year
21
MPOG 2019
• Retrospective single center
analysis (Wash U)
• Machine learning approach
to predictive modeling
• Perioperative electronic
data from > 95,000 patients
split into training, validation
and testing groups
• Deep learning model
provided greater predictive
power for 30-day
postoperative mortality than
conventional tools
22
MPOG 2019
And one more in
press for December
• Retrospective single center
analysis (UCLA)
• Machine learning approach
to predictive modeling
• Perioperative electronic
data from > 53,000 patients
• Deep learning model
provided greater predictive
power for in-hospital
postoperative mortality than
conventional tools
23
MPOG 2019
Summary
• BJA publishes clinical trials, data analytics,
laboratory studies, systematic reviews and meta-
analyses, audits, guidelines, case reports,
correspondence
• Prospective national audits have been major
drivers for practice guidelines
• Retrospective observational studies are being
driven by large data sets provided by health
system EHRs to identify associations and
develop risk prediction tools by machine learning
25
MPOG 2019
25
